Quantitative evaluation of effects of speech recognition errors on speech translation quality

This paper investigates the relationship between the quality of speech translation outputs and the errors in a speech recognition subsystem. In this study, we assume that a speech translation system is a sequential combination of speech recognition and automatic translation subsystems. We conducted speech translation experiments while changing parameters in the speech recognition subsystem to get different speech recognition results, which were then fed into the translation subsystem. We applied regression analysis to the interrelationship between the speech recognition outputs and the final translation results. We found that particular kinds of speech recognition errors, including deletion of punctuation marks and substitutions of nouns, cause severe semantic errors in speech translation. We also found that the final translation quality degrades logarithmically with respect to the number of speech recognition errors.